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Vision-Based  Leader/Follower  Tracking  for  Nonholonomic  Mobile 

Robots 


Hariprasad  Kannan,  Vilas  K.  Chitrakaran,  Darren  M.  Dawson  and  Timothy  Burg 


Abstract — This  paper  presents  a  strategy  for  a  nonholo¬ 
nomic  mobile  robot  to  autonomously  follow  a  target  based 
on  vision  information  from  an  onboard  pan  camera  unit 
(PCU).  Homography-based  techniques  are  used  to  obtain  rela¬ 
tive  position  and  orientation  information  from  the  monocular 
camera  images.  The  proposed  kinematic  controller,  based  on 
the  Lyapunov  method,  achieves  uniform  ultimately  bounded 
(UUB)  tracking. 

I.  Introduction 

Autonomous  guided  vehicles  (AGVs)  have  found  a  variety 
of  commercial  applications.  In  industry,  AGVs  are  utilized 
primarily  for  material  handling.  Typically,  such  vehicles  have 
utilized  laser/sonar  rangefinders,  optical  sensors,  or  magnetic 
guidance  systems  for  navigation  through  constrained  envi¬ 
ronments.  More  recently,  owing  to  the  increase  in  computing 
power  of  commercial  off-the-shelf  computers,  it  has  become 
possible  to  process  data  from  information  rich  sensors  such 
as  a  standard  video  camera,  for  guidance  and  navigation  of 
vehicles.  For  example,  vision-based  systems  for  autonomous 
motion  control  of  ground  vehicles  in  urban  environments 
have  been  developed  (e.g..  The  California  PATH  program 
[9]).  Research  in  this  area  also  has  implications  for  tech¬ 
nological  progress  in  assistive  and  rehabilitation  robotics  as 
well  [15].  In  addition,  many  applications  require  autonomous 
ground  vehicles  to  move  in  a  formation,  for  example,  explor¬ 
ing  remote  locations,  surveillance,  and  enclosing  a  target 
[14],  To  this  end,  various  approaches  to  formation  control 
and  utilizing  vision  in  navigation  have  emerged.  For  instance, 
[4]  investigates  a  graph  theory  based  approach  to  formation 
control.  A  vision-based  solution  to  formation  control  is 
presented  in  [3],  In  [12],  an  Extended  Kalman  Filter  is 
applied  to  visual  data  to  achieve  leader/follower  tracking.  A 
nonlinear  tracking  controller  utilizing  a  novel  sensor,  namely 
an  omnidirectional  camera,  is  presented  in  [17].  The  work 
in  [13]  demonstrates  a  Lyapunov-based  output  feedback 
approach  to  formation  control. 

In  this  paper,  we  present  a  Lyapunov-based  approach 
to  vision-based  leader/follower  tracking  for  nonholonomic 
mobile  robots  equipped  with  a  monocular  calibrated  camera 
mounted  on  a  controllable  pan  unit.  Such  an  approach  views 
the  formation  control  problem  as  a  combination  of  several 
similar  local  control  problems  requiring  each  follower  to 
track  its  assigned  leader.  The  tracking  objective  requires 
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information  related  to  the  leader’s  motion,  which  is  extracted 
from  the  video  stream  captured  by  the  follower  camera. 
This  type  of  camera-based  solution  eliminates  the  need  for 
any  sophisticated  communication  between  the  leader  and 
the  follower;  furthermore,  models  for  the  leader’s  motion 
and/or  the  leader’s  geometric  configuration  are  not  required. 
Specifically,  a  reference  image  is  initially  captured  when  the 
follower  is  at  the  desired  position  and  orientation  relative 
to  the  leader.  If  the  current  image  captured  by  the  follower 
coincides  with  the  reference  image  as  the  leader  is  in  motion, 
the  control  objective  is  achieved.  To  perform  this  visual 
servoing  task,  we  utilize  the  homography-based  approach 
(also  called  2^D  visual  servoing  [11])  to  quantify  motion  in 
Euclidean  space  by  utilizing  image  space  information  (i.e., 
image  coordinates  of  features  on  the  leader).  Along  this  line 
of  reasoning,  the  development  in  [1]  describes  an  algorithm 
to  track  prerecorded  images  of  a  fixed  target  generated 
by  a  nonholonomic  mobile  robot.  In  contrast  to  [1],  the 
proposed  controller  is  different  because  the  target  image  (i.e., 
the  leader)  is  moving;  furthermore,  it  must  be  emphasized 
that  the  motion  of  the  leader  need  not  be  nonholonomic. 
Therefore,  tracking  can  be  achieved  for  a  wider  range  of 
leader  motions.  As  one  might  expect,  unconstrained  motion 
on  a  plane  requires  a  vehicle  with  three  degrees  of  freedom 
(d.o.f),  two  for  position  and  one  for  orientation.  The  follower 
vehicle  that  we  consider  in  this  paper  is  an  underactuated 
mobile  robot;  the  robot  is  equipped  with  two  actuators,  one 
that  enables  forward  motion  and  another  that  enables  turning. 
Augmenting  the  robot  with  a  pan  camera  unit  provides  an 
additional  d.o.f  that  makes  the  robot-camera  system  fully 
actuated  with  respect  to  the  task  of  the  follower  tracking  the 
desired  (reference)  image  of  the  leader,  and  hence,  allows 
us  to  design  an  integrated  controller  that  maintains  the 
follower  robot-camera  system  at  the  desired  location  behind 
the  leader.  A  pan  camera  on  the  robot  also  has  the  additional 
advantage  of  maintaining  line  of  sight  with  feature  points  on 
the  leader  more  effectively  compared  to  a  body  fixed  camera 
(i.e.,  it  is  more  likely  that  the  target  will  remain  in  the  field 
of  view). 

II.  Problem  Formulation 

Let  us  denote  the  inertial  frame  and  the  frame  fixed  on 
the  mobile  robot  as  I  and  F ,  respectively.  The  x-axis  of  F  is 
along  the  forward  direction  of  the  robot.  The  robot  moves  on 
a  plane  parallel  to  the  XY  plane  of  F.  It  is  assumed  that  the 
origin  of  the  camera  frame,  denoted  by  C ,  coincides  with 
that  of  the  robot  frame  F.  The  camera’s  optical  axis  is  along 
the  x-axis  of  C.  The  frames  C  and  F  are  aligned  when  the 


pan  angle,  denoted  by  9p(t)  £  R1,  is  zero  and  the  camera’s  Given  the  arrangement  of  the  camera  frame  C  and  mobile 
optical  axis  is  along  the  approach  direction  of  the  robot.  The  robot  frame  F,  the  transformation  between  C  and  F  has  the 

following  form  [16] 


Rn  = 


cos  9n  —  sin  9V 


sin  9V 


cos  f 

0 
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feature  on  the  leader  is 


Fig.  1.  Relationship  between  coordinate  frames. 


As  shown  in  Figure  1,  the 
at  a  constant  position  Sj  £  R 
[  Xi(t)  yi{t )  Zi(t)  ]T  £  I 
the  same  feature  expressed  in  the  follower’s  camera  frame 
C.  Let  XqL  £  R3  and  RF  £  SO( 3)  denote  the  position  and 
orientation  of  the  leader  relative  to  the  follower  camera  frame 
C,  respectively  and  let  XqL,  £  R3  and  f?£«  £  SO( 3)  denote 
the  desired  position  and  orientation  of  the  leader  relative  to 
frame  C,  respectively.  At  the  desired  configuration,  let  to*  £ 
R3  denote  the  coordinates  of  the  ith  feature  relative  to  frame 
C. 

From  Figure  1,  it  can  be  seen  that  rrij  (f)  and  to*  are 
defined  as  follows 


relative  to  L.  Let  m,i(t)  = 
denote  the  coordinates  of 


frame  fixed  on  the  leader  is  denoted  by  L.  The  leader  moves 
on  the  same  plane  as  the  follower.  Let  L*  denote  the  fixed 
reference  position  and  orientation  of  the  leader  with  respect 
to  C,  captured  initially  by  the  follower  camera.  The  z-axis 
of  all  the  frames  are  parallel  to  each  other  and  orthogonal  to 
the  plane.  Figure  1  shows  the  spatial  relationships  between 
the  various  frames.  The  goal  of  this  paper  is  to  design  a 
controller  that  will  move  the  follower  and  the  PCU  such  that 
L  coincides  with  L*  as  seen  from  the  camera  frame  C . 

In  this  paper,  we  will  use  the  following  convention  [6]: 
vectors  are  specified  as  v™b,  which  signifies  the  vector  v 
of  the  6  frame  relative  to  the  e  frame  and  expressed  in 
the  n  frame,  'x' ,  'v'  and  'to'  denote  position,  translational 
velocity  and  angular  velocity,  respectively.  A  rotation  matrix 
is  specified  as  R ^  £  SO( 3),  which  transforms  coordinates 
defined  in  the  /  frame  to  the  t  frame. 

A.  Geometric  Model 

The  mobile  robot’s  linear  and  angular  velocities,  denoted 
by  vfF(t),  uifF(t)  £  R3  respectively,  can  be  defined  as 
follows 

vfp  =  [  vp  0  0  ]T 

cofF  =  [00  toF]T 

where  vF(t ),  u>F(t)  £  R1.  The  angular  velocity  of  the  leader 
with  respect  to  I  and  the  angular  velocity  of  the  PCU  with 
respect  to  F,  denoted  by  ioFC{t)  £  R3  respectively, 

are  expressed  as  follows 

=  [  0  0  Wi  ]J 

ojfc  =  [  0  0  uic  ]T  (2) 

wc  =  9P 

where  ujp(t),  LOc(t)  £  R1. 


mi  —  XCL  A-  RLsi  (4) 

mi  =  Xcp.+Rp.Si-  (5) 

Eliminating  Sj  from  (4)  and  (5),  we  obtain 

TOi  =  x  +  Rm*  (6) 

where  R(t)  £  SO(3 )  and  x(t)  £  R3  are  rotation-  and 
translation-like  variables,  respectively,  defined  as  follows 

R  =  RclRc  (7) 

x  =  xcL-RxcL*-  (8) 


We  assume  that  at  least  four  feature  points  tracked  on  the 
leader  are  selected  such  that  they  are  coplanar.  Let  n*  £  R3 
denote  the  constant  unit  normal  from  the  origin  of  the  camera 
frame  to  the  feature -plane,  when  the  leader  is  at  L*.  Let 
d*  ^  0  £  R1  denote  the  constant  distance  to  this  plane  from 
the  camera.  Then  d*  is  the  projection  along  n*  of  all  the 
features  to*,  which  can  be  expressed  as  follows 

d*  =  n*Tm*.  (9) 

Using  (9),  (6)  can  be  re-written  as 

(R+^n*T)m* 

A  d  l  GO) 

H 

where  H(t)  £  R3x3  is  an  Euclidean  Homography  [8]. 

Remark  1:  If  it  is  not  possible  to  utilize  four  coplanar 
features  on  the  leader,  H[t)  can  still  be  constmcted  using 
the  virtual  parallax  algorithm  [11].  However,  in  this  case,  at 
least  eight  non-coplanar  features  are  needed. 


B.  Euclidean  Reconstruction 


We  need  to  extract  Euclidean  related  position  and  orien¬ 
tation  information  from  the  images  of  the  features  on  the 
leader.  The  coordinates  of  the  feature  points  in  the  camera 
images  are  represented  as  2D  homogeneous  coordinates, 
denoted  by  Pi(t),p*  £  R3,  relative  to  frame  C.  The  pin-hole 
camera  model  [8]  relates  these  to  the  normalized  Euclidean 
coordinates  m*/.V  and  as  follows 


Pi  =  A p*  = 

r L  X X* 


(ID 


where  A  £  R3x3  is  a  known,  constant  and  invertible  intrinsic 
camera  calibration  matrix  and  xRt),  x*  £  R1  are  the  x- 
coordinates  of  m,i(t)  and  to*,  respectively.  Hence,  image 
coordinates  can  be  introduced  into  (10)  as  follows 


Pi  = 


AH  A  y  p 


Oti 


G 


(12) 


where  the  depth  ratio  -Aj-pj  is  denoted  by  a,(f)  £  R1  and 
G(t)  £  R3x3  is  a  full-ranked  homogeneous  collineation 
matrix  [8].  Given  a  set  of  corresponding  points  (pi(t),  p*), 
the  elements  of  G(t)  can  be  determined  up  to  a  scale 
factor.  Since,  there  are  eight  unknowns,  a  minimum  of  four 
corresponding  pairs  are  required  to  compute  G(t).  Since, 
the  intrinsic  camera  calibration  matrix  A  is  assumed  to  be 
known,  H(t)  can  then  be  uniquely  determined  from  G(t). 
Homography  decomposition  algorithms,  described  in  [5],  [8], 
can  be  used  to  calculate  R(t)  and  j^x{t)  from  H(t).  Note 
that  it  is  possible  to  obtain  only  -hx(t)  and  not  x (t). 


IIL  Control  Development 

The  control  objective  is  to  ensure  that  the  image  of  the 
leader  as  seen  by  follower  camera  matches  the  reference 
image  captured  when  the  leader  was  at  a  desired  position  and 
orientation.  Referring  to  Figure  1  and  the  previous  section, 
this  objective  can  be  framed  as 

xcl  -*  xcl*  >  rl  rl*  as  t  — >  oo.  (13) 


Examining  equations  (7)  and  (8),  it  is  seen  that  the  above 
control  objective  is  achieved,  if  R(t)  — >  and  x(t)  — >  0 

as  t  — »  oo,  where  Inxn  denotes  an  n  x  n  identity  matrix. 

We  develop  the  control  inputs  Veit),  cujr(i) ,  toc(t)  based 
on  the  measurable  signals  R(t),  x(t)  and  &p(t).  To  fa¬ 
cilitate  the  control  development,  we  make  the  following 
assumptions 

Assumption  1:  The  pan  angle  of  the  PCU  (i.e.,  6p(t))  is 
measurable  using  suitable  sensors  like  encoders.  Hence,  we 
can  compute  R^(t)  using  (3),  which  is  required  in  the  control 
development. 

Assumption  2:  The  kinematic  variables  of  the  leader  are 
bounded  i.e.,  XjL(t),  vPL(t),  u>jL(t)  £  £oo-  Since,  there  is 
no  communication  between  the  robots,  these  signals  are  not 
available  to  the  controller  on  the  follower  robot. 

Assumption  3:  The  mechanical  dynamics  of  the  follower 
robot  and  the  PCU  can  be  neglected. 


Assumption  4:  The  follower  robot  is  always  behind  the 
leader  and  the  features  on  the  leader  are  always  within  the 
field  of  view  of  the  follower’s  camera. 


A.  Open-loop  Error  System 

The  orientation  error,  denoted  by  eg(t)  £  R3,  is  defined  in 
terms  of  the  axis-angle  representation  of  the  rotation  matrix 
R(t),  given  in  (7),  as  follows  [16] 

eg  =  pcj)  (14) 

where  p(t)  £  R3  represents  the  unit  axis  of  rotation  and 
4>{t)  £  R1  denotes  the  angle  of  rotation  about  p,(t)  (—7 r  < 
<j>(t)  <  7r)  and  are  computed  as  follows 

cf>  =  cos”1  (§(tr(-R)  —  i))  =  (15) 

where  S  (•)  £  R3x  i  denotes  the  skew-symmetric  matrix  form 
of  a  three  element  vector  [16].  Given  that  the  mobile  robot 
moves  on  the  XY  plane,  p(t)  and  tj>(t)  take  the  following 
simplified  form 

M=[0  0  1  ]T  (/>  =  QCl  —  Ocl*  (16) 

where  Oij(t)  £  R1  is  the  right  handed  rotation  about  the 
z-axis  that  aligns  the  orientation  of  frame  i  with  frame  j 
(we  assume  —n  <  6ij(t)  <  t r).  Hence,  for  our  system  only 
the  third  component  of  eg(t ),  denoted  by  egz{t)  £  R1,  is 
non-zero  and  has  the  following  form 

eg  =  [  0  0  egz  ]  ^7^ 

egz  =  Ocl  —  Ocl*- 


This  form  of  e$(t)  is  useful  for  control  design.  By  taking 
the  time  derivative  of  (14),  the  open  loop  dynamics  of  the 
orientation  error  can  be  expressed  as  follows 


eg  —  L^ql 

=  —LuRp^ujfc)  +  L^RpUjfp 


(18) 


where  Lw(t)  £  R3x3is  a  Jacobian-like  matrix  given  as 
follows 


-  $SM  +  (i  -  Bjfj)  s<»)2 

sinc{4>)  =  ^1. 


(19) 


The  derivation  of  Lu(t)  is  presented  in  detail  in  [10].  Using 
(1),  (2)  and  (16),  the  expression  in  (18)  can  be  simplified  as 
follows 


eg=  0  0  -uf-uc+wl  •  (20) 


The  position  error  is  defined  in  terms  of  the  measurable 
signal  4 rx(t)  from  the  vision  system.  We  first  define  an 
auxiliary  variable  a 'hit)  £  R3  as  follows 

xh  =  c^x-  (21) 

By  taking  the  time  derivative  of  (8),  the  open  loop  dynamics 
of  Xh(t)  can  be  obtained  as  follows 

Xh  =  ( xcl ~  R  xcl *)  • 


(22) 


For  further  development,  we  utilize  the  following  properties 
of  general  rigid  body  motion  [16] 

Kb  =  Rbveb 

Rl  =S^lb)Rl  =  RtS^beb)  U3) 

where  e  and  b  can  be  any  two  frames  of  reference.  Referring 
to  Figure  1,  x^L{t)  can  be  expressed  as 

XCL  =  RFRf  (xil  ~  xif)  (24) 

and  its  time  derivative,  by  using  (23)  and  some  manipulation, 
can  be  expressed  as 

XCL  =  RI  {VIL  —  RfvIf)  —  R{uw)xCL-  (25) 


stability  analysis,  the  control  inputs  vp(t),  u>F(t)  and  u>c{t) 
are  designed  as  follows 

vf  =  kve  i 

u>f  =  -ie  2  (34) 

u>c  —  u>f  +  keegz 

where  kv .  ku,  kg  G  JR1  are  positive  tunable  gains.  Given 
these  control  inputs,  the  closed-loop  dynamics  of  ei(i),  e2  (t) 
and  egz(t)  can  be  expressed  as  follows 

egz  =  —kgegz  +  wl 

d*e  i  =  —  kve\  —  hUJd*e%  +  d*^n  (35) 

d*e 2  =  d*kueie2  -  kud*S0e2  + d*£i2- 

IV.  Stability  Analysis 


After  substituting  (25)  into  (22)  and  also  using  (23),  we 
obtain  the  following  expression 

d*xh  =  -RcfVif  -  d*R$S  {ujKj)  R^xh  +  6 
a  =  R?v1l-S(u?l)Rx  cl. 

where  (i  £  l3  contains  the  unmeasurable  quantities.  Con¬ 
sidering  the  form  of  (26)  and  utilizing  Assumption  1,  we 
can  define  the  position  error  for  our  system,  denoted  by 

Xh(t)  £  R3,  as  follows 

xh  =  Rcxh ■  (27) 


The  open-loop  dynamics  of  the  position  error  can  be  deter¬ 
mined  by  taking  the  time  derivative  of  (27)  as  follows 

xh=  Rcxh  +  Rcxh ■  (28) 


After  substituting  (26)  into  (28),  and  utilizing  (23),  the 
following  expression  can  be  obtained 

xh  =  ~-^vfF- S(wfF)xh  + ^R%£ i  (29) 

a  =  ^a-  (30) 

Given  that  the  mobile  robot  and  the  leader  move  on  the  XY 
plane,  Xh(t)  and  £i(i)  have  the  following  form 

Xh  =  [  Xhi  xh2  0  ]T 
Ci  =  [  ?n  ?12  0  ]T 

where  xhi(t),  xh2{t),  £n(t),  £i2 (t)  G  K1.  After  using 
(1),  the  open  loop  dynamics  of  Xhi(t)a\\A  Xh2(t)can  be 
expressed  as  follows 

xhl=  +  WFXh2  +?ll 

-  -  ,7  (32) 

xh2=  —UFXhl  +  sl2* 

B.  Closed-Loop  Error  System 

First  we  define  auxiliary  error  signals,  denoted  by 

ei(i),  e2  (f)  £  l1,  as  follows 


ei  —  Xhi  +  So 
e-2  =  xh2 


(33) 


where  #o  £  l1  is  a  positive  design  constant.  Based  on 
the  open-loop  dynamics  (20)  and  (32),  and  the  subsequent 


Theorem  1:  The  follower’s  translational  and  rotational 
velocities,  and  the  pan  velocity  of  the  camera,  designed  in 
(34),  guarantee  uniform  ultimate  boundedness  (UUB)  of  the 
position  error  signal  Xh(t)  and  the  orientation  error  signal 
eg  {t)  in  the  neighborhood  of  zero  in  the  following  sense 

||*/i(t)||  ,  || ee (t) ||  <  ai  exp(-a2t)  +  a3  (36) 

where  ait  a2.  a3  G  K1  are  variable,  positive  constants. 

Proof:  We  choose  the  following  non-negative  scalar  func¬ 
tion  as  the  Lyapunov  candidate 

V=\e2ez  +  ^el+^el  (37) 

After  taking  the  time  derivative  of  (37)  and  substituting  the 
closed-loop  dynamics  for  eg(t),  e±(t)  and  e2(i)  from  (35), 
the  following  inequality  can  be  obtained 

V  <  -ke20z  -  ke\  -  kd*Soe\  -  kgie20z  +  \eez\  \Pi\ 
-kvlel  +  d*  |ei|  \p2\  -  fca,id*60e|  +  d*  |e2|  \p3\ 

(38) 

where  k,  kg± ,  kv\ .  k^i  G  K.1  are  positive  constants  such 
that  kg  =  k  +  kgi,  kv  =  k  +  kv i,  ku  =  k+  ku i,  and  we 
have  utilized  Assumption  2  to  upper  bound  to  Lit),  £n(f), 
£12  it)  by  positive  constants,  denoted  by  pi,  p2,  P3  G  K1,  as 
follows 


\uL\  <  Pi 

£lll  <  P2 

(39) 

£l2  <  Pz- 

By  using  Young’s  Inequality  [7],  the  last 

six  terms  of  (38) 

can  be  upper  bounded  as  follows 

\egz\  (|/3i|  —  kgi  \e$z\) 

< 

m\2 

koi 

d*\ei\m-^\ei\) 

< 

d*2  m2 

kv  1 

(40) 

d*  |e2  i\P3\  -  k^iSo  |e2|) 

< 

d*m2 

kuj  i^o 

From  (38)  and  (40),  V it)  can  be  further  upper  bounded  as 
follows 


V  <  —ke0z  —  ke\  —  kd*Soe\  +  e  (41) 

where  e  =  +  d  — h  dk  ^  G  K1.  We  can  upper- 

bound  aj/j(f)and  eg(t),  by  solving  the  above  differential 
equation,  as  follows 

||5ft  (f)||  ,  ||ee(f)||  <V(t)<  exp (-a2t)  +  a3  (42) 


robot  trajectory 


where  011,0:2,0:3  G  R1  are  positive  constants  defined  such 
that 


01 

02 


O3 


1/(0) 

min(fc,  kd*So) 
max(i,f ) 

£ 

(y.2 


(43) 


The  form  of  (36)  follows  directly  from  (42). 

We  now  verify  the  boundedness  of  all  signals  in  the 
system.  From  (37)  and  (41),  eg(t),  ei(t),  62(f)  €  Coo-  Hence, 
from  (33),  Xh{t)  £  C 00 >  and  from,  (27)  Xh{t)  £  Coo-  It  can 
be  seen  from  (8)  and  (21)  that  x^L(t)  £  Coo-  From  Assump¬ 
tion  2  and  (24),  xjF(t )  £  Coo-  Also,  based  on  the  control 
design  in  (34),  u>fF(t),  uFC{i)  £  Coo-  Consequently, 

from  (20),  eg(t)  £  Coo-  From  (18),  x>fc(t)  G  Coo ■  Therefore, 
ih(t)  G  Coo  (see  (26))  and  from  (28)  Xh  (t)  £  Coo- Hence, 
all  signals  in  the  system  remain  bounded  during  closed-loop 
operation. 


V.  Simulation  Results 

We  developed  a  simulation  for  validation  of  the 
leader/follower  algorithm  in  C++  utilizing  QMotor  [2],  a 
real-time  control  development  environment  for  the  QNX 
Operating  System.  Four  coplanar  feature  points  at  the  corners 
of  a  square  fixed  vertically  on  the  leader  formed  the  set  of 
tracked  features.  For  the  simulation,  the  image  coordinates 
of  these  feature  points  as  seen  by  the  camera  on  the  follower 
were  obtained  based  on  the  kinematics  of  the  vehicles  and 
the  camera.  The  homography  matrix  was  decomposed  into 
its  constituent  motion  parameters  using  an  implementation 
of  the  algorithm  given  in  [5].  The  leader  moved  in  a  circular 
trajectory  on  the  XY  plane  and  its  motion  was  not  nonholo- 
nomic  because  its  feature  plane  was  always  parallel  to  the 
y-axis  while  in  motion.  The  initial  positions  of  the  follower 
and  the  leader  were  set  as  x\F  =  [  8.5  —0.5  0  ]  and 

x\L  =  [  10  0  0  ] T,  respectively.  The  desired  position  of 

the  follower  was  at  1  meter  from  the  leader  (i.e.,  XqL,  = 
[l  0  0  ]  ) .  The  initial  and  desired  orientation  angles  of 

the  follower  with  respect  to  the  leader  were  20  degrees  and 
0  degrees,  respectively.  The  trajectories  of  the  leader  and  the 
follower  are  shown  in  Figure  2.  The  evolution  of  position 
and  orientation  errors  with  time  is  shown  in  Figure  3.  The 
velocity  control  inputs  remain  within  bounds  as  shown  in 
Figure  4.  For  getting  practical  results,  the  magnitudes  of 
the  follower  linear  velocity,  angular  velocity  and  the  PCU’s 
angular  velocity  were  saturated  at  0.85  m/s,  0.3  rad/s  and 
0.75  rad/s,  respectively.  Also,  a  soft-start  function  multiplied 
each  of  the  velocities,  so  that  their  values  were  zero  when 
the  simulation  began  and  then  ramped  up  to  the  designed 
velocities.  The  following  control  parameters  were  selected 
upon  trial  and  error 


kg  =  30,  kv  =  10,  ku  =  30,  (44) 

S0  —  1  x  10-3. 


The  follower  took  20  seconds  to  attain  the  desired  motion 
because  of  the  initial  error  in  position  and  orientation.  It  was 


Fig.  2.  Trajectories  of  the  Leader  (o,  dashed)  and  the  Follower  (o,  solid). 


position  error  (x)  position  error  (y)  orientation  error 


Fig.  3.  The  Position  Errors  (dimensionless)  and  the  Orientation  Error 
(radians). 


observed  that  the  follower  traced  a  circle  while  maintaining 
the  desired  offset  from  the  leader,  which  was  the  expected 
result. 


VI.  Conclusions 

This  paper  presented  a  vision-based  controller  for  a  non- 
holonomic  mobile  robot  with  an  onboard  pan  camera  to 
follow  a  leader  vehicle  autonomously.  Homography  based 
techniques  were  used  to  obtain  relative  position  and  orienta¬ 
tion  information  from  the  monocular  camera  images,  and  the 
controller  design  was  based  on  the  Lyapunov  method.  The 
proposed  controller  was  shown  to  achieve  uniform  ultimately 
bounded  (UUB)  tracking.  Currently,  we  are  working  on  the 
experimental  verification  of  the  proposed  algorithm. 
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